Skip to content

Conversation

@compilade
Copy link
Contributor

#822 (by @mofosyne) has introduced a naming convention for GGUF model files, but the way it names k-quants doesn't follow the established practice (all other places where k-quants are named use Qn_K where n is the number of bits per weight excluding the scales).

rg -i 'KQ\d' doesn't return anything related to quants except for this recently-added section, while
rg -i 'Q\d_K' returns a lot of things related to k-quants when run in ggml and llama.cpp repos

So this renames KQ2 to Q2_K, for consistency. This should avoid unnecessary confusion.

(note that the recently-added wiki page about "tensor encoding schemes" will need to be updated too, since it is the only other place I found to also use this KQ<X> naming scheme)

@ggerganov ggerganov merged commit 8d6b703 into ggml-org:master May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants